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ABSTRACT 

The utility of augmenting displays to aid the human operator in 
controlling high order complex systems is well known, ^nalyt ical 
evaluations of various display designs for a simple k/s ' plant in a com- 
pensatory tracking task using an Optimal Control Model (OCM) of human 
behavior is carried out. This analysis reveals that significant 
improvement in performance should be obtained by skillful integration of 
key Information into the display dynamics. The cooperative control syn- 
thesis technique previously developed to design pilot-optimal control 
augmentation is extended to incorporate the simultaneous design of per- 
formance enhancing augmented displays. The application of the coopera- 
tive control synthesis technique to the design of augmented displays is 
discussed for the simple k/s ' plant . This technique is intended to pro- 
vide a systematic approach to design optimally augmented displays 
tailored for specific tasks . 
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I . INTRODUCTION 


With the advent of high perf orinance aircraft , the amount of infor- 
mation to be processed by the pilot to successfully accomplish the 
assigned task has increased tremendously • It has ^ therefore, become 
critical to determine and limit information to the best informational 
set needed by the pilot so as to reduce his workload and improve his 
performance by reducing complex, unusual tasks to simpler, familiar 
ones. The need for providing augmented displays to the pilot to achieve 
this objective is very well understood. In the present paper, analytl- 
cal^evalua tion of various display "quickening” control laws for a simple 
k/s' plant is carried out. The evaluation is done for a tracking task 
using an Optimal Control Model (OCM) [1] of human behavior. 

A methodology to design pilot-optimal display/control augmentation 
systems which analytically takes into account the control and informa- 
tion processing limitations of the human controller is proposed. This 
methodology is an extension of the cooperative control synthesis tech- 
nique previotisly developed to design pilot optimal control augmentation 
[2,3,4]. Though the proposed methodology has been developed so as to be 
applicable to simultaneous synthesis of pilot optimal control augmenta- 
tion and display augmentation, the present discussion focuses on the 
application of the technique to display design only. 

The cooperative display design technique is applied to synthesize 
performance enhancing augmented compensatory displays for the k/s plant 
in the tracking task. The displays thus obtained show improved tracking 
performance for much reduced mean square pilot iDput when evaluated 
using the OQt. Ttoreover, the methodology offers considerable potential 
as a tool for providing a systematic approach to task tailoring of aug- 
mented displays. 


IT. DISPLAY DFSICN FOR k/s PLANT 

2 

Consider the k/s plant dynamics as discussed by Kllenman et al. in 
(1|. The system state equations are 




or in concise form 
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( 2 . 1 ) 


Here k=l in. /in. and the state x^(t), a first order Markov process 
aving a break frequency of 2 rads/sec, is the velocity of the command, 
(t) has intensity = 0.217 to give L{x^} = 0.054 in. 
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e: 


error 


= X] + = e: error rate (2.2) 

where the pilot is assuned to he able to reconstruct the error rate by 
ohservinp the error itself. For the OCM model, the pilot's cost func- 
tion is taken to be 


J(u) = + rE{u^} (2.3) 

where ”r*’ is chosen so as to pive a neuromuscular lap tine constant, 

T = 0. 1 secs. 

N 

For all tlie analysis carried out in this section, the followinp 
parameters were set for the OQf pilot model 


a. Pilot's observation time delay set to 0.2 seconds 

b. Observation noise ratio was set at -20 dB 

c. Motor noise ratio was set at -25 dB 

d. The weighting on the control rate in' the pilot's cost function 
was always adjusted to yield =0.1 secs. 

e. Very low values of thresholds were used for the observations 
made available to the pilot . 

With the above parameter settings, the OCM analysis of system 
(2.1)-(2.B) gave results that are compatible with those given in [1]. 
These results are as sho\^ in the last row of Table 1 . 


Next consider the display dynamics having the form 


’‘d ■ "d’‘d + "d 

with the display quickening control u, given by 

d 




(2.4) 


(2.5) 


where y^ is the vector of plant outputs which are available for driving 

the display and 0 is the set of display control gains being determined, 
d 

or 


^d 




( 2 . 6 ) 


The dynamics of the display a\igmented system can then be written as 



A ‘ 
o I 




(2.7) 
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The pilot's observations for the display auf^inented systen are 



( 2 . 8 ) 


where it is again assumed that the pilot is able to reconstruct the rate 
of display by observing the displayed variable itself. The pilot's per- 
formance obiective for the display augmented system is to minimize the 
cost 


J,(u) = E{xJ} + rF.{u^} (2.q) 

d d 

Witli the above formulation in mind, the performance of the display 
augmented system is evaluated using the model for various valties of 

a a^nd various combinations of the display control gains G . T\>70 cases 

or y are considered. The first is when the display state Is driven only 
by the error, i.e. state x , and the second is when y^ consists of both 
the error as well as the plant velocity state x^. 

Case(a) : 

This is the simplest possible case for display of the form (2.4). 
For this case the displayed variable is iust lagged error. The display 
dynamics are given by 

♦ 

’’d ■ •■'d’'d + '■•d2’‘2 

where g^ is the display control gain on state x^ . For = -a^ (2.10) 
can be written in transfer function form as 

-a 

X^(s) = ^ xAs) (2.11) 

d s-a, 2 

d 

Since x^ = e, it is clear from (2.11) that in the steady state the 
displayed variable will closely approximate the error. 

The OCM results for various values of a are presented in Table* 1. 
The results of Table 1 are also plotted in Fig. 1 and Fig. 2, and 
correspond to the curve marked (T) . From these plots it is clear that 
with only error driving the display, the pilot's performance is worse 
than the Idealized no-display case. As a^ > -<», the pilot's performance 
approaches that of the case with no display augmentation. The no- 
display case wl)ich then corresponds to an infinetly fast display is not 
desirable because of the Inherent limitations on the pilot's ability to 
perceive fast changing signals, and the need to provide filtering of the 
noisy outputs. It might be reasonable to select a display which has a 
sligl^itly liigher bandvjidth than the pilot, so a^ in the range -10 to -20 
sec is desirable since the pilot's minimum neuro-muscular lag time 
constant is approximately 0.1 secs. 

“ T 

Case(b) y^ = x^] : 
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For this case, the display dynamics have the form 
where i = 2, 3 is the display gain on the state x.. 

fii 1 

Since the plant velocity state, tlie above form of display 

w^ill provide lead information to the pilot. The pilot's performance can 

then be expected to improve as the gain g,_ is increased. 

di 

jOCM analysis is carried out for two values of a : -10 and -20 

sec • For each of these values of a^ , varied from 

1 to b in steps of 1. The restilts of this analysis are presented in 

Table 2 and are also plotted in Fig. 1 and Fig. 2 so as to compare them 

with the case of g^^ = 0. Ty the two figures, the curve marked 

corresponds to a = -10 sec and that marked © to a , = -20 sec 

d d 

Fig. 1 is a plot of mean square error vs. mean square control rate 
(u) for the various display cases discussed above and Fig. 2 is a plot 
of mean square error vs. the mean square control input (u). The point 
marked A corresponds to the no display case in the two figures. From 
these two figures it is clear that the mean square input and the mean 
square control rate both decrease as the display control gain g^^ is 
increased. What is most interesting is that the mean square error ini- 
tially decreases as g is Increased and then starts increasing beyond a 
certain value of g that depends on the choice of the display bandwidth 

and the display gain g^ 2 * booting that earlier work [5,6] has shown that 
the pilot's v;orkload is directly related to the mean square control 
rate, this means that it is possible to improve performance (of which 
mean square error is a measure) while at the same time decreasing 
pilot's workload and the control energy required by a skillful integra- 
tion of key Information into the display dynamics. Moreover, the 
results Indicate that for a given display bandwidth there is an optimal 
choice of display control gains v/hich leads to the best possible perfor- 
mance. For Instance, in Figures^l and 2, point C is such an optimal 
display design for a^ = -20 sec , and for this case the performance is 
slightly better than the no-dlsplay case. Meanwhile the pilot's work- 
load and the control effort required are both significantly reduced. 

It then appears desirable to develop a systematic approach to 
display augmentation wluch will make it possible to directly synthesize 
the optimal display design without having to resort to trial and error. 
In the following sections an extension of the optimal cooperative con- 
trol synthesis technique is proposed as a methodology to synthesize 
pilot-opt imal display/ control augmentation systems . 
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TABLE 1: OCM BRSULTS FOR VARYING DISPLAY BAJnTO*JIDTH (p; = -a,) 

n2 d 


( sec ) 

r 

(t = 0.1 secs) 
N 

M.S. 
Error 
(in. ) 

M.S. 

Input 
(in. 3 

M.S. 

Control rate 
(in. /sec ) 

-3 

A. 6x10“ 
5.Rxl0 ^ 

0.0215 

2.176 

106. PI 

-10 

0.0177 

1 .543 

76.47 

-20 

6.2x10 
6.25x10 
6.3x10 ^ 

0.0157 

1.353 

67.44 

-30 

0.01A2 

1 .261 

62. P 

-100 

0.0135 

1.223 

60. P8 

NO DISPLAY 

7.0x10 ^ 

n.ni3i 

l.lAl 

54.73 


TABLE 2: OCV RESID.TS FOR VARYING DISPI.AY GONTROL GAINS 




= -IQ, 8 


a 

CJ 

C 

CM 

1 

II 

JO = 20 

33 

d 

“ Ti.'s.”^ 

(sec ) ^ 

M.S. 

12 

M.S. 

~ 7 

(SecV 

M.S. 

d2 

M.S. 


Error 

Input 

Control rate 

Error 

Input 

Control rate 



(tYl3 

(inVsec^) 


(ivi.n 

(lYi.^/sec^) 

1 

0.0144 

1.113 

54.75 

0.014 

1.175 

58.06 

2 

0.0138 

0.733 

35.92 

0.013 

0.P68 

47.47 

3 

0.0143 

0.486 

23.71 

0.0127 

0.789 

38.49 

4 

0.0157 

0.33P 

16.52 

0.0128 

0.639 

30.97 

5 

0.0175 

0.248 

12.05 

0.0131 

0.521 

25.15 i 

6 

0.01P5 

0.187 

P.Ol 1 

0.0136 

0.427 

20.46 


optimal gooperative coi-jtrot V dispt.ay design M EHODOLOGY 

PROBLEM FORMULATION: 

In this section the mathematical formulation of the cooperative 
control synthesis technique is presented, and necessary conditions for 
the simultaneous optimality of the display and control aiigmenta t ion sys-^ 
terns are developed. The procedure followed here is very similar to that 
of [3, 4]. 

Consider tlie dual controller system described by the linear time 
invariant set of first order differential equations 

x^Ax+B, u_-fB u +Dw (3.1) 

o Jo 1 2o 2 o 

__ n __ _ ^2 

with xeR , ^tid w a zero-mean Gaussian white noise process 

V7i til intensity The two controls represent two physically independent 
controllers . 

The display dynamics are assumed to be of the form 




B, 11 

do ci 


(3.2) 
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^ IMj 

with display quickening controller. The 

ohiective is to find the optimal cooperative controllers 1 and 2 
(Uj and U 2 ) alonf^ witli tlie optimal display control law u^. 

Controller 1 has noisy observations available for feedback 

given by 


= C, X + 
10 


^dl^d 


C u 
u d 


+ V 


(3.3) 


where v is also a zero-mean Caiissian white noise process with intensity 
V . ^ 

y 


The augmentation controller u^ ^r»d th^ display control law are 
assumed to have noise-free system outputs y^ and y^ , respectively, 
available for feedback, where 


^2 = 




f -- 




(3. A) 


Note that the above formulation does not allow feedback of the display 
states into the augmentation controller u^* 


Finally, tl^ese two controllers are constrained to have the direct 
output feedback form 


"2 ■ '^2>'2 ■ SSo* 


“d ■ '=d>'d ■ "d'^d I - 


(3.5) 


which is consistent with the desire for simple, easy to implement con- 
trol laws. 


The interaction between the different controllers is shown in the 
block diagram of Figure 3. 


DFSTCN OBJRCTIVFS: 


Controller 1 is to be optimal with respect to the cost 

T 

J = Fflim /(>^^0 x+x,0 X -tuyR u -f-u^F u )dt} (3.6) 

1 ^ T*' lo d Id dill/]/ 

T^® o 

in the presence of the action of control inpxits u^ and u^ . Here F{*) 
indicates the expected value operator and the weighting matrices are 

"lo ' "■ "id > "• '’l > "• ''l > "■ 

Conversely, Controller 2 (n^) ^nd the display control law are to 
be optimal with respect to the cost 

T 

J 2 = F.dlu, 1 (3.7) 

t-^» o 
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in the presence of the control action u . 

^ ^ N > 0, F > 0, F yn. 
ics (3.1) with the display dynamics ^3.2), 
of this augmented system is obtained to be 


The weighting matrices are 
Augmenting the system dynam- 
the state-space description 


p 

X 1 

7 7^' 


A I 0 ’ 
_0| 



+ 

^1 

lo 

u, + 

o I 

CM 1 1 

4* 

» r 

O 1 
) L 

u + 

D 

o 

[*< 1 . 


< 

c 

1 

/d 


0 

1 

["J 

2 

[”«oJ 

d 

0 


w (3.8) 


with 


Defining x “ (^* ^d^* (3.^) can be written in a compact form 

appropriate definitions for the matrices as 


X = Ax + B^Uj + B^u^ + B^u^ 4* Dw 
The outputs can similarly be written as 


(3.D) 


n ° Sx V'<l * "y 

yj - lc-2^ "lx ' V 

The two cost functions can then be expressed in terms of the aug- 
mented state vector x 

T 

J, = Fflim / (x"^F) X + + uTf u )dt} 

T-voo o ‘ 1 J 1 1 z 

T 

J 2 = F{llm 1 g?V+ujR2U^+ujF2U2+T7jF2^u^)dt} (3.11) 

7-^00 0 


wViere the. weighting matrices 0^ and are appropriately defined. 
SOLII'T'ION FOP u^ : 

Tn the presence of the action of control inputs u^ and u^ , as given 
by (3.5), the dynamics of the augmented system are obtained to be 


X = A X + B u, + Dw 
atig 1 1 

y = C X + V 
1 aug y 


(3.12) 


where 


A (A + B/..C + B n C ) 

aug = 222 d d d 

c - (c, + c c c ) 


auR = 1 

and the performance Index becomes 


n d d ' 


(3.13) 


T 

1 f/-T 


.1, = niin^ (3.14) 


T„ - 


7^00 
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Equations (3.12) and (3.1A), in the case of uncorrelated process 
and measurenent noises and for V > D, describe the standard non- 
sinp,ular linear quadratic Caussian regulator problem, b^hen stabiliza- 
bili ty and detectability conditions for the system are satisfied, the 
optimal controller is known [7] to have the form 


"i ■ hx 


(3.13) 


where x is the minimm mean-square estimate of the system state vector 

X* 


'the Rain matrix is given by 

k) = (3.1b) 

with P > 0 the symmetric solution of the algebraic Ricatti equation 

(3.17) 

1 i 1 

The dynamics of the state estimator are 




X ■ + "l“l + b'Xl - 

where the Kalman filter gain matrix is given by 

T 

M = Z C V 
1 aug y 

with Z > 0 the symmetric solution of the algebraic equation 

A E + E - Ec"^ v'^^c E = 0 

^^og aug aug y aug 


(3.18) 


(3.19) 


(3.20) 


SOLETION FOR u„ AND u,: 

2 d 

The optimal controller Uj as derived above has the form 


AjX + Mjyj 


(3.21) 


X 

where A — (A +B k -M 0 ). 

1 = aug 11 1 aug 

Then in the presence of the control action , the system dynamics 
can be written in terms of the augmented state vector q ~ COL (x> X) 


(3.22) 


which can further be v;rit ten in a compact form with appropriate defini- 
tions of matrices as 


• 





^d 


'p ' 0 ' 
1 

I 

1 w 

q = 


a 4- 


"2 

■’l^u 

"d 

0 1 >■ “ 
L 1 U 


fv-j 

. y J 


/ / / 

q = Ajp + ^d''d 


(3.23) 
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The intensity of the process w' is W' = 



The index of perforinance then becomes 


T-voo o 


(3.24) 


O- ' 0 

0' A --1 


The_desipn ohiective can then be stated as to find the optimal con- 
troller and optimal display control u, which minimize the cost J as 
piven by r3,24). 


Proceedinp, in a way as detailed in [4], it can be shown that the 
pains and which correspond to the simultaneous optimality of the 
two controllers Ti- and ij, are piven by 

/ d r T r -I 


T T 

C C 

r., - oi,„,/nc^|ol 


(3.2'^) 


2d M,C 
1 n 


([Cdlo] l^H)- 


(3.26) 


--T 


vrlth L = Efq q } satlsfyini* the relation 


A L + LA" + D'W'D''^ = 0 
c c 


and H satisfyinp 


(3.27) 


AH + HA + n = n 
c c 

where the follov/inp definitions have been used 

i ' "‘'‘I A 

Ar -..-c-r-A,-' 0- 

1 auH' J 

I 


(3.28) 


30.10 



Though the metliodolop.y developed above is applicable for simultane- 
ous synthesis of optimal control and display atigmenta tion , only the 
application to display design will be discussed in this paper. For the 
case of display design only, the controller u^ is inactive and the sys- 
tem dynamics and corresponding conditions for optimality are accordingly 
simplified. 

TV. APPLTCATIOK OF DT SPLAY DFSION MF.THOnOT.OOY TO k/s^' PLANT 

A computer code was developed to determine tlie optimal display con- 
trol gains using the above methodology. The details of a similar com- 
puter code are documented in [4]. The algorithm is iterative using a 
gradient search technique. Civen a starting display gain matrix (includ- 
ing the null matrix) the. display gains that satisfy the conditions of 
optimality as stated in Section III are determined. 

2 

The dynamics of the k/s , plant augmented with the display, are as 
in Section II. The application of Jhe methodology to optimal display 
design for the case of a = ~20sec ^ with both x^ and x^ driving the 
display will be discussed. 


The controller u^ is analogous to tlie control rate u of the OQ!, 
so in order to be consistent with t)ie pilot's stated objective of regu- 
lating the display, the cost is defined as 

•j, = + R,F{uh (A.l) 

1 dll 

where R, is chosen so as to satisfy the requirement of T,, = 0.1 seconds 
for the pilot s neuro-miiscular lag. Also the process noise and the 
measurement noise in the problem formulation are chosen such that the 
controller for the beginning display dynamics is compatible with the 
OCM model corresponding to the dynamics. (The reader is referred to [4] 
for details of how to achieve this). 


The cost is defined as 


J., = o F,{e^} + P„F,{uh + F„,F{u 2} 
z e Z J Zd d 


(4.2) 


v/hich is reflective of tlie overall objective of reducing the tracking 

error through the means of an "intelligent" display. Note that in 

(4.2), needs to be positive definite in order to get a finite 

optimal solution to the problem. However, since the display control 

does not reflect any measure of energy, the weighting F may be chosen 

small such that its contribution to the cost J is not significant. For 

the results presented in this section F = O.nOl was used. 

Zd 


The results obtained using the optimal cooperative design methodol- 
ogy for various values of and presented in Table 3. For all 

these cases the starting display gains were taken to be = [20, 0]. 

In "^able 3 the optimal display gains are listed as x^7ell as the results 
of evaluation of the corresponding augmented dynamics using the OCM. 

The parameters that define the OC'l were set to the values stated in Sec- 
tion II. The analysis results for the no-display case and the cases 


30,11 



of = [ 20 , 0 ] and == [ 20 , 3 ] for = ~ 20 sgc“*\ are also listed in 
Table 3 to provide a comparison. The results of Table 3 are also plot- 
ted in ^ip,ures 4 and 5. 

Note that as the relative weiphtinp on the error is increased in 
the cost the optimal cooperative display deslpn methodology does 

lead to display pains which pive improved performance at the expense of 
increased control activity. Tims tins methodolopy, throuph a proper 
choice of weiphtinps in the cost function nrovides a systematic 

approach to desipn of task- tailored display 'aupmentation. 

Also note that for all the 5 cases of display desipn using this 
methodology, the final optimal display gains were such that the perfor- 
mance is significantly improved as compared to the beginning display and 
at the same time the workload (u) and control effort (u) are consider- 
ably reduced. If the weighting on the error is made high enough (cases 
4 and 5), performance comparable to the no display case and the best 
case corresponding to G^ =r [90, 3] of Section II is obtained for signi- 
ficantly reduced workload and control effort. Moreover it is clear that 
for the display bandwidth such that a = - 20 sec > performance better 
than that of case 5 cannot be obtaineo. Increasing the weight on error 
in the cost function any further would only liave the effect of lead- 
ing to a display design requiring higher control effort without any 
noticeable Improvement in performance. 


TABLE 3: nCM BEEULTS F0K_nPTIMAL niRPLAYS FOR k/s^ PLANT 

= - 20 sec , 0 ^ = ^^ 3 ! 


S.N. 


^2 

Optimal G^ 

Error 

Input ' 

Control Rate 

1 


1 

20x1 0^^ 

[36.6, 11.9] 

O.OIA 

0.389 

18. 6A 

2 


2 

20x10 1 

[A9.2, 13) 

0.0132 

0.A92 

23.72 

3 


2 

10x10 Z 

[6A.A, 16.3] 

0.0131 

0.51A 

24.78 

A 


A 

10x10“^ 

[88.1, 17.3] 

0.01272 

0.650 

31.58 

5 


A 

5x10 

1 n 

18.2, 22.6] 

0.01269 

0.665 

32. AO 

A 


NO 

DISPLAY 


_ ■ 

0.0131 

1.141 

54.73 

B 

BEG. DISPLAY FOR DESIGN 

[20, 0] 

0.0157 

1.353 

67.44 

C 

BEST PER 

. DISPLAY (II) 

[20, 3] 

0.0127 

0.789 

38.49 


COrCLNSTON S 

Through ocr analysis of a simple k/s' plant it was sbo\sm that the 
performance of a human controller can be improved and bis workload sig- 
nificantly reduced by providing him an active display which integrates 
information for tlie system dynamics being controlled. A methodology 
based on the optimal cooperative control synthesis technique was sug- 
gested as a means to synthesij^e optimal display gains, tailored for 
specific tasks. The application of this methodology to the k/s"^ plant 
was discussed and tlie results presented show that the methodology has 
potential for providing a systematic approach to display design. 
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The results obtained for the k/s^ plant need to be experimentally 
verified with man in the loop simulation in order to validate the 
display design methodology. Research in the area of applying the pro- 
posed design methodology to high order dynamical systems in a complex 
mul ti -control task scenario is presently ongoing and the preliminary 
results are quite encouraging. 
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NOMENCLATURJE 

Only driving display, - -5. -10, -20, -50, -100 
A - no display 

© ®d ' -^0* «d2 ■= ^d3 “ 2- 3* '*• 5. 6 

" - ^d3 = 2 

@ - -20, - 20, g^2 = 1* 2. 3, A, 5, 6 

- «d3 “ 3 

FIG, 1 PERTORMAKCE VS WORKLOAD 
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NOMENCLATURE 


Only driving display, = -5, -10, -20, -50, -100 
A - no display 

= -10, gj2 ^ ^0* 6 j 3 ■= 1. 2, 3, A, 5, 6 
B - g^3 - 2 

@ a^ = -20, g^ » 20, g^3 ■= 1. 2. 3, A, 5, 6 
C - g^3 » 3 

FIG. 2 performance VS CONTROL EFFORT 
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NOMENCLATURE 

A: NO-DISPLAY 

B: - (20. 0) 

C; ■= (20, 3] 

1: = 1. = 20 X 10 

2: Q = 2, R- •= 20 X 10 

e 2 

3: Q *= 2, = 10 X 10 

e 2 

4: Q « 10 X 10 

e 2 

5: Q - 4, R. - 5 X 10 

2 

FIG. 4. PERFORKAKCE VS VORICLOAD FOR OPTIMAL DISPLAYS 







NOMENCLATURE 


A: NO -DISPLAY 

B: G^ = [ 20 . 0 ] 

C: Gj - [20. 3] 

Qg - 1, R2 = 20 X 10 

2: “ 2. = 20 X 10 

3: Qg - 2. R^ = 10 X 10 

Qg “ Rj “ 10 X 10 

5: Qg “ 4. Rj - 5 X 10~ 

I^IO- S . PERPORMANCE VS CONTROL EFFORT FOR OPTIMAL DISPLAYS 
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